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ASSESSING AND CONTROLLING THE AVAILABILITY 

OF Failure-Degraded Service Agents 



by 

D. P. Gaver and P. A. Jacobs 
EXECUTIVE SUMMARY 

Military items such as airborne surveillance systems (UAVs, JSTARS, helicopters, 
etc.) or combat vehicles (tanks, APCs, ships) may have high effectiveness when available 
on station, but require occasional restoration (refueling, re-arming, scheduled 
maintenance) and repair after unscheduled failures of certain subsystems. This 
requirement takes them off station, where delays occur that are affected by the numbers 
and types of support resources and the philosophy of scheduling those resources. 

This paper considers the effect of decision choices on long-run item availability on 
station when items can be in several levels of capability/effectiveness when on station. 
The model is used to show that a simple binary decision rule (that depends on ratios of 
endurance, failure, and restoration and repair rates) guides the decision as to whether a 
failed item should be completely repaired to its highest level, or returned to duty at an 
incomp letely-capable state. 

View this as an indicator of the types of rules anticipated to apply in realistic 
generality. These will be the subject of additional research. 
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1. Introduction and Problem Formulation 

In a variety of situations several individual items (agents) coexist and perform 
operational functions, such as region surveillance or actual combat. They require 
occasional support (e.g. if vehicles, or military “platforms”, refueling and repair). Time 
out for restoration (refueling) is inevitable and unavoidable, but failure of a unit’s 
functional capability or operational condition, while necessary, need not be complete: a 
partially-failed or degraded unit can profitably remain active under some circumstances. 
Under certain circumstances, to be described, it is overall-effective to partially repair a 
completely failed item, rather than expend time and resources for repair to complete 
functionality. A unique military situation is that in which an item has been subjected to 
chemical or biological attack, and must be decontaminated: there can be a choice between 
partial, or complete decontamination', see Klutke and Novikov (1996). In this paper we 
provide initial models and rules for the control of all such systems. 

Examples of the options and decision choices occur extensively in the military: 
depending on need for information, it may well be more effective to quickly field 
partially effective sensor (e.g. UAV) systems than to wait for complete repair. The same 
can be true of tactically required artillery pieces, tanks, or reconnaissance or attack 
aircraft; the choice of partial or total decontamination is another example, mentioned 
above. Other examples occur when prompt assistance is needed after an emergency, 



either natural (e.g. earthquake or fire) or terrorist-caused. Failures may now be injuries to 
humans or infrastructure; partial “repair” means initial diagnosis and stabilization of an 
injured individual, or provision of temporary alternative communication or transportation 
service. 

The situation modeled is analogous to the classical repairman problem (see Feller 
(1968)) but is more general - and realistic - in that, for us, the classical “machines”, our 
agents, can fail partially, i.e. to one of several intermediate levels of productivity and 
then, having finally reached the restoration-repair service facility, can be repaired there to 
chosen levels. This control feature is, in principle, accomplished probabUisticaUy in our 
models. The control objective is to maximize the expected utility of the system of agents, 
using as control options the rate of demand for repair, and the choice of repair objective. 
The utility function involved is a linear function of the numbers of agents available in 
each performance category. For UAVs this can be expected time on station, weighted for 
sensor capability. 

Our analysis begins by proposing a quasi-stochastic “expected value” or deterministic 
model for a system of agents that can each be in one of three states of performance 
capability: Good, Medium, and Bad. When restoration or repair is required the agent 
progresses to a central homebase server at which accumulated and queued restoration and 
repair tasks are processed in a weighted processor-sharing manner (this is unrealistic in 
detail, but is analytically tractable; it will be useful for guiding more detailed 
simulations). Control is accomplished by choice of the weighting probabilities. This 
model accounts for restoration-repair facility saturabUity by using an approximation 
described in Filipiak (1988). The deterministic model is especially convenient for 
exploring time-dependent or transient system behavior. 

Next, a Markov stochastic network version of the above model is constructed using 
the techniques of Kelly (1979). The implications of the two model types are compared 
numerically and seen to be in useful agreement for the utility function adopted. 
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Our work is related to policies for minimal repair; cf. Brown and Proschan (1983), 



Phelps (1983), Chen and Feldman (1997), Kijima, Morimura, and Suzuki (1988) and 
Righter (1997). Also relevant are analytical models of sortie generation, Hackman and 
Dietz (1997) that address more complex setups in a steady-state context. 

2. A Pseudo-Stochastic Deterministic Expected-Value Model 

The system will be described in terms of the following state variables. For the present 
model these are real-valued and positive functions of time; in the subsequent stochastic 
model they are state variables of a multivariate Markov process in continuous time. 

Let 

Uc{t) = number of agents in Good condition and Up (productively available) at time r; 

Uw(t) = number of agents having experienced a degrading failure and Up at time r, 
such agents are said to be in Medium condition; 

Dcait) = number of agents that have experienced a mission-affecting failure (MAF) 

while in Good condition and that are Down for repair at time t\ such an agent 
is totally unproductive and is said to be in Bad condition; it will be repaired to 
the Good state; 

DmbO) = number of agents that are Down at time t because of a mission affecting 
failure from the Up Medium state; 

a decision opportunity: a fraction a^c are sent for further repair to Good, 

1 - otMC are released to Up Medium state; 

DcR(,t) = number of agents in Good condition that are Down for restoration (refueling 
etc.); they will be restored to the Good state; 

Dm(.t) = number of agents in Medium condition that are Down for restoration to the 
Medium state at time t; 

a decision opportunity: a fraction a\tR are sent for further repair to Good, 

1 - ctMR are released to Up Medium state; 

DmcU) = number of agents Down in Medium condition after minimal repair, waiting for 
repair to the Good state at time t. 
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Parameters: 



N 

^MB 

Vsc 

Vsm 

Wr{G) 



Total number of agents (e.g. mobile sensors) 

rate of occurrence of a degrading failure for a Good Up system 

rate of occurrence of a total failure for an agent (e.g. a sensor) in Good 
condition 

rate of occurrence of a total failure for an agent in Medium Up condition 

mean endurance time for an agent in Good condition 

mean endurance time for an agent in Medium Up condition 

mean restoration time for an agent in Good Down condition 

minimal mean restoration time of an agent in Medium Down condition; agent 
wUl remain in Medium condition 



\l}im{B) = minimal mean time to repair an agent that has experienced a fatal failure; an 
agent in Medium condition wUl still be in Medium condition after this failure 

l/Mm(M) = mean time to repair an agent for a degrading failure 

ctRG = fraction of service effort applied to repair a restored Medium Down agent to 
Good 

olmc = fraction of service effort applied to repair a Medium Down agent that had a 
fatal failure while it was in Good condition 

Although the above are constants in the developments that follow, they can be 
replaced by nearly arbitrary functions of time, or even state, especially in the quasi- 
stochastic deterministic model The differential equations arising in that formulation can 
be numerically solved, e.g. using MATLAB. This paper is devoted to the steady-state 
situation, for which analytical solutions are available, and which permit derivation of 
(almost) optimal control strategies. But transient and also time-dependent solutions are 
often important and can be obtained more easily with the present approach than using 
other analytical tools of which we are aware. 

Let S{t)=DcR{t)+DMR{t)-\-DGB{t)+DMB{t)+DMG{t), the number of agents down at time t. 
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Let H{x) be a differentiable non-increasing function of x; it will represent the amount 
of service effort when there are x agents waiting for or being served. See Filipiak (1988). 
An example is 



H{x)^- 



1 



1+c^x 

where Cs is a suitable constant (unity in what follows). 
Time-Dependent Equations 



Ut ' V ' 

Failure or Restoration demand from Good state 

>- ^ 

Service to Good state 



( 2 . 1 ) 



(2.2a) 



“^GA/^gW ” (^M+^A/s)^Af(0 

at ' — — ' ' v < 

Partial Failure, Failure to Bad or 

Good to Medium Restoration demand from Medium 



(2.2b) 



+H(5W)[m. (M)(1 - ccgo)Dm(‘) +Mm(5)(l -a«c )O«fi(0] 

> 

Restoration or Repair to Medium 



dDcit{t) 

dt 



= SoUc{t) -H{S{t))iir{G)DcR{t) 



Good Restoration 
demand 



Restoration 
completion to Good 



£«£(«(() -H(S(t))^XM)DnK{t) 

at 



Medium 

Restoration demand 



Restoration completion 
to Medium 



at ' ' V ' 

Medium fails Repair of Medium, failed to Bad, 

to Bad to Medium 



(2.2c) 



(2.2d) 



(2.2e) 



at . ' V V ' 

Number continuing from beyond minimum repair: Repair to Good completion 

Medium-repaired to Good repair state 
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= AgsUg (<) - H{S(t))^lJB)DaB (<) (2.2g) 

at ' ' ' y ' 

Total failure. Repair Down Good to Good 
Good to Bad 



In Appendix 1 it is shown that the solution to the steady-state equations obtained from 
(2.2) by setting derivatives equal to zero is 



Um=U-1/{1 + k) 



(2.3) 



Ug=U-k/{\ + k) 

where U is the modified solution of a quadratic: 

2N 



U = 



[l+{\+c,N)c/{i + K)]+^[l + (l + c,N)c/(l+K)f-4CsNc/(l+K) 






bAM) IIJB) 



^G , ^GM , ^GB 



A, 



^r{G) 



2.1 Repair Control for the Pseudo-Stochastic Model 

Suppose the expected utility of the system of agents is written as 



(2.4) 

(2.5) 

( 2 . 6 ) 

(2.7) 



g(cc-,p,N)^UGioc.N)+pUM{a,N) 

where we let anc = ocmg = cc, the probability that an agent that has been Down and has 
been restored/repaired to Medium level is allocated further repair to the Good state, with 
0<P< 1. This notation makes explicit the degradation of capability associated with 
being in the Medium rather than Good state, and also the control inherent in the 
probability cc, finally y the influence of the agent population size, N, is expressed. 

Given values for P and N in addition to other rate parameters, the utility function g 
can be numerically studied/optimized on a. Considerable numerical exploration, using 
random selections of system parameters in a manner analogous to that suggested by 
Miller (1997), leads to the empirical conclusion that random strategies (0 < a< 1) are 
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rarely optimal, and, when they do occur are only slightly better than pure strategies 
(a = 0, or a = 1). In fact, consideration of the case N large clarifies the situation and leads 
to the nearly-optimal control policy as follows: 



M.(M) ^JB) 

b- ^G , I ^GB 

pi,{G) fljM) ^ijB) 

^GM 

Then K= ya and c = a + bya, the parameters in the solution U, 



When N becomes large. 

Optimum a = 1 if P< ajb\ 
Optimum a = 0 ifp>afb. 



To show this, let N — > «»» in expression (2.5) for U to obtain 

c/(co) = i±il = -i±2^ 

c,c c,{a + bya) 



so, by (2.3) -(2.4) 

(1+jc) (1 +k:) 

ya + P 
Cs{bya+a)' 



g{a;p,oo) = 



l + K 



Since 



dg ^ 7{a-bP) 
da {bya + of" 



( 2 . 8 ) 

(2.9) 

( 2 . 10 ) 

( 2 . 11 ) 



the above policy is verified. 



3. A Stochastic Model 

Consider a closed queuing network with N agents (e.g. mobile sensors). There are two 
service classes: up (U) and down (D). An agent chooses a type each time it leaves the 
down service class in good condition. The agent types are 
G: is in good condition 
M: is in medium condition 

The expected time an agent of type i spends in the U class is 

— = - - r - 

+ ^GM + ^GB 

Let aaiU) be the expected amount of time an agent spends up in good condition before it 
is reclassified; let aM(U) be the conditional expected amount of time an agent spends up 
in medium condition before it is reclassified, given it spends time in Medium condition; 
let a(U) be the total expected time an agent spends in the up state before it is reclassified. 
For 0<a«G+«MG 



aM(U) = 



e^+X 



MB 






£Af +A 



•<3^^G + 



^MB 



MB 



^M+^MB 



<^MG 



(3.2) 



aG{U) = 



1 



^G'^^GB'^^GM 



(3.3) 



a(U) = aa(U)+ , au(U). (3.4) 

^G'^^GB'^^GM 

Let ac{D) be the expected amount of time an agent spends down with a MAF but is in 
otherwise good condition before it is reclassified; let be the conditional expected 
amount of time an agent spends down in Medium condition before it is reclassified, given 
it spends time in Medium condition; let Qmr{D) be the conditional expected amount of 
time a Medium condition an agent spends in restoration before being reclassified, given it 
spends time in Medium condition; let OMMiD) be the conditional expected amount of time 
an agent in Medium condition spends being repaired for MAF before being reclassified. 
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given it spends time in Medium condition; let a{D) be the expected amount of time an 
agent spends in Down state before it is reclassified. For Q<aRc+aMc 






''MB 






£m 



(^RG + 



''MB 



MB 



£m+^ 



(^MG 



1 

^m{M) 



(3.5) 



MB 



ac(D) = 



^G 



1 



■+• 



''GB 



1 



a{D) = aciD) + 



<^mr{^)-' 



^G+^GB+^GM f^ri^) ^G+^GB~^^GM 
^GM 



^G + ^GB + ^GM 

^M ]__ 

^M "^^MB Mr(^) 



‘^m{D) 



^M 



/Y I /Y 

^RG +“: — — <^MG 






^AZ+'^MB ^AZ+'^AfB 

-^AZB ^ 

^M + ^MB PhJM) 



^M 



^M'^^MB 



^MB 

<^RG + T^; <^MG 



£m 



MB 



(3.6) 

(3.7) 



(3.8) 



(3.9) 



Theorem (3.12) of Kelly (1979) implies that the limiting distribution of the number of 
agents in the up class and the down class is 






a{Uf^ a{D) 



''D 



nu 



I 



(3.10) 



J][min(/:,s) 

/r=l 



where K is a normalizing constant and s is the number of servers in the D class. The 
expected number of agents in the up class (respectively down class) is 



N 



E[U]^Ku X«t/ 



ri(j=Q 

N 



a{U) 

nu\ 



”c/ 



\riD 



n alDY 

(respectively E[D] = Kd ) 

]^min(/t,s) 

k=\ 



(3.11) 



(3.12) 
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where Ku and Kd are normalizing constants. The expected number of agents in the up 
class that are in good condition (respectively medium condition) is 



£[C/o] = £[J/]^ (3.13) 

(respectively, £[(/«] = £[t/] °“^^ ) (3.14) 

The expected number of agents in the down class that are being repaired for an MAP but 
are otherwise in good condition (respectively are in medium condition) 

£[£>c]=£[0]^^ (3.15) 

(respectively, £[£>«] =£[Z)]^^) (3.16) 



Numerical Examples 

Below are tables comparing the numbers of agents in various categories for the 
deterministic and stochastic models. While the two models are not exactly comparable, 
the deterministic model yields results close to those of the stochastic model 

Table 1 

N=4; Ac;a^= 0.1; /l<;fi=0.05; Ams=0.05; £< 3 = 0 . 2 ; €m= 0.2; fim(M)=0.5; /im(G)=0.5; fi„iMG)=5; 
^r(G)=lO; fir(M)=lO-, a/j(3=0.5; aMC=0.5; single server; Cj=l 

Expected Number 



Category 


Deterministic 


Queuing 




Model 


Model 


Good: Up 


1.81 


1.87 


Medium: Up 


1.45 


1.50 


Good: Catastrophic Failure 


0.32 


0.27 


Medium: Catastrophic Failure 


0.25 


0.21 


Repair: Medium to Good 


0.06 


0.05 


Good: Restore 


0.06 


0.05 


Medium: Restore 


0.05 


0.04 
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Table 2 

N=4; XcM=l; A<;b=0.05; Xm=0.l; £g=0.2; e«=0.5; /i„(M)=0.5; /i„(G)=0.5; ^„(MG)=5; 
lir(G)=\0; iir{M)=l\ aRc=0.5-, aMc=0.5; single server; q=1 

Expected Number 



Category 


Deterministic 


Queuing 




Model 


Model 


Good: Up 


0.52 


0.60 


Medium: Up 


1.73 


2.00 


Good: Catastrophic Failure 


0.14 


0.11 


Medium: Catastrophic Failure 


0.95 


0.76 


Repair: Medium to Good 


0.29 


0.23 


Good: Restore 


0.03 


0.02 


Medium: Restore 


0.34 


0.27 



4. The Repair Policy for the Stochastic Model with N=1 

Let 



E[U{a)] = 



1 



-+ 






1 1 



^CB + £c + Acw CC Xmb +£« 



(4.1) 



E[D{a)] = 



1 



£c 



1 



XcB +£c XcB +£g +AgW fJ-r{G) 



, Agm 


1 1 


^M 1 1 


Xmb 1 


I 

Aca +^G +^GAf 


ji„{M) a 


_A«fl + £m Mr(A^) 


XmB+£m 



(4.2) 






1 1 



gia;p)= 



.^GB + £c + Agm Aa/S + £m 



E[U{a) + D{a)] 



a + P 



'-CM 



Xgm +f M 



a 



j 1 Acs 1 £c I Agm 


, ^CM 


^M 1 Xu 


ll^B) flr{G) Ai.(M)_ 


1 

XuB + £m 


MM) 



(4.3) 



ct + P — 

r 



Note that 



dg(a;P) 


r ^ fill 




da 


( 1 Y 

a{\+b]+-a^ 



(4.4) 



Hence the repair policy is as follows. 

The Repair Policy for Stochastic Model with N=l: 

K ^ then the maximizing a is 0=1; always repair the agent in Medium 



condition to Good 
a 



If p> 



1 + b 



then the maximizing a is o=0; never repair the agent in Medium 



condition to Good. 
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APPENDIX 1 
Long-Run Solutions 



Setting the derivatives equal to zero in system (2a) - (2g) results in the following 
equations 



r (Al-la) 

^H{S\^r[G)DGR+ llm{^)^MG +/^/ti(^)^Gs] 

0 = ^GM^G 

r T (Al-lb) 

+H (S)[//;. (M)(l -ccrg)Dmr + P-m (-S)(l ~ ^mg)^mb\ 

0 = SgUg -H{S)^,{G)Dgr (Al-lc) 

0 = SmUm - H{S)^,{M)Dmr (Al-ld) 

0 = ^MB^M -H{S)fijB)DMB (Al-le) 

0 = H{S)[^ri^)ai^GGMR + f^mi^)(^MG^MB]~ ^{S)P'mi^)GMG (Al-lf) 

0 = ?LgbUg-H{S)piMDgb (Al-lg) 



These must be solved subject to the condition that aU state values sum to N. Equations 
(Al-ld), (Al-le), and (Al-lg) result in 



Gmb - 



_ ^MB^M 






r. _ 

UkAJ^ — 



B>gb - 



^GB^G 



(Al-2) 

(Al-3) 

(Al-4) 
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Substitution of (Al-3) and (Al-2) into equation (Al-lb) results in 



Thus 



0 - ^GM^G ~ A/ + yj M 



Ug=^M 



where 



- ^RG^M '^^MG^MB 

^gm 



(Al-6) 



(Al-7) 



if A<3 m > 0. 

Substitution of (Al-3) and (Al-2) into equation (Al-lf) results in 

0 = £M^RG^M'^^MB^MG^M~^i^)f^mi^)^MG (Al-8f) 

Multiplying equation (Al-ld) by fir(G)lfJ.r(.M), equation (Al-le) by equation 

(Al-8f) by fj.r(G)/}J.m(M) and equation (Al-lg) by fJ.^G)IHm(B), adding the resulting 
equations and using (Al-6) results in 



where 



Note that 



c£/„ = H(S)S 



c = 



11,{G) ^r{M) pijB) fi^{B) 



(Al-9) 



(Al-10) 



S = N-{1 + k)Um (Al-11) 

Thus (Al-9) can be rewritten as 

cUm =[N-{1+k)Um]H{N-{1+k)Um) (Al-12) 

Putting U=Uc+UM=(i+K)UM and substituting into equation (Al-12) results in 
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= [M- U]H(N - U) 



Rewriting, 



c U 
\ + k[N-U] 



H{N-U) 



Equation (Al-14) has one solution between 0 and N. 

In the special case H(x)=\/(^\+CsX) equation (Al-14) is a quadratic 



0 = -^U^-il + -^(\ + c,N))U + N 

\ + K l-f-KT 



U = 



a+Y7^(i+^.A^))- 



V 



(l+:^(l-hc,;V))2-4 



1-1- K" 



CsC 
1 + K 



N 



r. CfC 
2—^ 
l + K 



(AM3) 



(Al-14) 



(Al-15) 



(Al-16) 



Finally 



Um=U/(1+i^ 



and 



Ug = 



K 

l-i-TC 



U. 
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APPENDIX 2 

A Repair Policy for the Deterministic Expected- Value Model 
Let aRG=ccMC=cc. Put 



£m , 

a = — : h- 



MIH) n.{B) 



Eg , Acw 

0 = T-T + 



ICB 



r = 



Ai.(G) 



^CM 



Note that K^ya and c=a+bya. Thus, 



c a+byoc 

= — = a- 

l + K 



y{b-a)a 
1 + yoc 



1+ya 

Let U(a) be the solution to equation (Al-9). 

Consider the weighted (expected) number of agents up 

g(a-.p) = Uo(a)+fiUu(a) 

_7a±P: 



1+ya 



U{a) 



(A2-1) 

(A2-2) 

(A2-3) 



(A2-4) 



(A2-5) 



for 0 < < 1 . 

Policy I: This maximizes a utility function that assigns a relatively low value to units in 
the Medium state. 

If bca, then g(a;p) is maximized at a=l; always repair an agent in Medium condition 
to Good. 

If b>a, then there is a 0 < )9b </?i < 1 such that for fi<Po g(a;{5) is maximized at o=l 
(always repair a Medium agent to Good); for P>P\ g{cc,P) is maximized at Of=0 (never 
repair a Medium agent to Good). 



18 



Policy II: This maximizes the long-run total number of agents up 

If b<a, then U{a) is maximized at a=l; that is always repair an agent in the Medium 
condition to Good. 

If b>a, then U{a) is maximized at a=0; that is never repair an agent in Medium 
condition to Good. 

Proof for Policy II: 

Note that f(cc) = is a decreasing function of a if b<a and is a nondecreasing 

l + K 

function of a otherwise. Since H{x) is a decreasing function, H{N-U) is an increasing 
function of U. Further, (N-U)H(N-U) is a decreasing function of U. If b<a so that /(a) 
decreases with a, then equation (Al-13) implies that U(a) increases with a. If b>a, then 
/(a) increases with a and U(a) decreases with a. Policy I follows. 

Proof for Policy I: 

Note that 



If b<a, then 
If b>a, then 



da ^ l+ya ^ l+ya da 

l+ya l+ya da 



l+ya 



-jU{a)+ 



dU{a) 

da 



^ j 3)>0; thus the maximizing a for g{a;p) is o?=l. 

da da 



dU{a) 

da 



<0. If /3=0, then j{a;0)=UG((x). Recall that 



Thus, 



Uc{a) = yaUM(a). 



-^Ucia) = yUM{cc)+ya-^U[4{cc) 
da da 
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and 



-^U{a) = ^Uc(a) + ^Uu(a) 
da da da 



= ->UM{a)+[Ta + l]—UM{a). 



Hence, 



d -^U(a)-yU^(a) 

-UM{a) = -^ < 0 . 

da ’ ya^l 

The value of a which minimizes Um((x) is 0=1. In addition -^Uj^{a)<-^U(a). Since 

da da 

4 ~V(a) = -^Uc(.a)+-^Uu(a) it foUows that 0 < -|-C/(a) = -f-Ua(.a). 

da da da da da da 

Thus, the maximizing value of a for Ug{ol) is o=l; always repair agents in Medium 

condition to Good. Thus, 



l + )0 



1+70 da 



Recall from Policy II that if j8=l then — g(o;l )<0 and the maximizing a is o =0 (never 

da 

repair a Medium agent to Good condition). Policy I follows. 
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